This report presents the evaluation approach developed for the DARPA Big Mechanism program, which aimed at developing computer systems that will read research papers, integrate the information into a computer model of cancer mechanisms, and frame new hypotheses. We employed an iterative, incremental approach to the evaluation of the three phases of the program. In Phase I, we evaluated the ability of system and human teams ability to read-with-a-model to capture mechanistic information from the biomedical literature, integrated with information from expert curated biological databases. In Phase II we evaluated the ability of systems to assemble fragments of information into a mechanistic model. The Phase III evaluation focused on the ability of systems to provide explanations of experimental observations based on models assembled (largely automatically) by the Big Mechanism process. The evaluation for each phase built on earlier evaluations and guided developers towards creating capabilities for the new phase. The report describes our approach, including innovations such as a reference set (a curated data set limited to major findings of each paper) to assess the accuracy of systems in extracting mechanistic findings in the absence of a gold standard, and a method to evaluate model-based explanations of experimental data. Results of the evaluation and supporting materials are included in the appendices.

可变形的图像注册，估计不同图像之间的空间转换，是医学成像中的重要任务。许多先前的研究都使用基于学习的方法进行多阶段注册来执行3D图像注册以提高性能。但是，多阶段方法的性能受到不受单个空间尺度上复杂运动的接收场的大小的限制。我们提出了一个新的注册网络，结合了递归网络体系结构和相互注意机制，以克服这些局限性。与最先进的深度学习方法相比，基于递归结构的我们的网络达到了肺计算机断层扫描（CT）数据集的最高精度（肺部的骰子分数为92 \％，肺平均表面距离为3.8mm ），这是腹部CT数据集中最准确的结果之一，具有9个大小的器官（骰子得分为55 \％，平均表面距离为7.8mm）。我们还表明，添加3个递归网络足以达到最新结果，而没有明显增加推理时间。